Data compression is widely used in modern data communication, transmission, and storage systems. The basic aim of data compression is to encode (or compress) a message into an equivalent, but shorter, compressed message. The compressed message can then be transmitted or stored using fewer bits and, when need, can be decoded (or uncompressed) to recreate the original message.
At the most fundamental level, data compression methods are characterized into two distinct classes: (i) lossless compression and (ii) lossy compression. In lossless compression, the process of encoding, then decoding, a message will always produce a perfect recreation of the original message—i.e., there is no data loss, hence the name lossless compression. With lossy compression, on the other hand, the process of encoding, then decoding a message, will not always produce an exact copy of the original message. Instead, the compression/decompression process may result in certain errors, or data loss. Such errors are typically small, and hopefully imperceptible by the user of the data. MP3 is a good example of lossy compression: Although the process of encoding, then decoding, a song using the MP3 standard will significantly alter the original bit stream, the user will not perceive a problem, since the song still “sounds” the same. Generally speaking, lossy data compression is used for sound and image files, whereas lossless data compression is used for text and data files.
In data compression, the term “compression ratio” refers to the ratio of the compressed message size to the uncompressed message size. Thus, a compression ratio of 0.5 implies that the compressed message is half the size of the original message.
The basic idea behind all data compression algorithms is to identify and exploit patterns in the data to be compressed, and to apply the greatest compression to the most commonly appearing patterns. For example, if the data to be compressed is ASCII text that represents written English, most people would realize that the most commonly appearing character is “e,” whereas characters like “x” and “z” are relatively uncommon. ASCII represents all characters using 7-bit patterns. While 7 bits represents the fewest number of bits that can be used to assign fixed-length codes to all characters, one can achieve better compression of English text by varying code lengths, and assigning shorter codes to the most common characters (e.g., 2 or 3 bits for an “e”) and longer codes to the least common characters (e.g., 9 or 10 bits for an “x” or “z”). Utilizing the well-known Huffman algorithm to optimally assign variable-length codes to ASCII characters results in a compression ratio of about 0.59 for English text. This, however, is still far from optimal, since the best known compression algorithm achieve compression ratios of about 0.2 for English text.
To key parts of any data compression algorithm are: (i) the model and (ii) the coder. The model component predicts the likelihood that a given symbol (e.g., a character) will appear in the message. In the ASCII English text example discussed above, the model was “context independent” because, for example, it predicts that “e” will be the most likely character, regardless of where one is in the text. More complex and powerful data models are “context dependent” because they may consider the context (e.g., what character(s) immediately precede the current character) in predicting what the next character will most likely be. For example, if one is encoding English text, a context dependent model would predict that the most likely character to follow a “q” would be “u,” rather than “e.”
In the data compression field, context dependent models are typically characterized as “first order,” “second order,” “third order,” and so on. A first order model is one that considers the preceding symbol in predicting the current symbol. A second order model considers the last two symbols. And a third order model considers the last three symbols. (Note, a “zero order” model is the same as a context independent model.) Because computational complexity grows super-exponentially with the order of a model, low-order data models (e.g., second or third order) are typically used in practice.
With a data model in place, the coder component then uses the symbol probabilities predicted by the data model to assign a code to each possible symbol. Here, the well-known Huffman algorithm can be used to produce an optimal assignment of codes (i.e., shortest codes for most common symbols, etc.) for a given data model. Alternatively, one can employ well-known arithmetic coding techniques, which are often superior for highly repetitive data streams.
Over the past decade, so-called PPM-based algorithms have achieved the best overall compression performance (i.e., lowest compression ratios). However, PPM-based algorithms tend to be slow. An “order-k” PPM algorithm uses an order-k context dependent data model, but with a twist. The twist occurs when the algorithm encounters a k-length context that it has never been seen before; in this case, it attempts to match the shorter (k−1)-length sub-context using a (k−1)-order data model. Attempts at this “partial matching” continue, using successively lower-order data models, until either (i) a sub-context is partially matched or (ii) a zero-order (i.e., context independent) data model is used.
The speed problem with PPM-based algorithms stems from the fact that the number of potential order-k PPM data models grows as the powerset of k. Hence, even for a modest value of k, choosing an appropriate PPM data model from among the vast number of alternatives can pose a computationally intractable task. Traditional PPM algorithms compute a new data model for each message to be encoded, thus not allowing the cost of the computationally-expensive model building task to be amortized over many compression/decompression cycles.
In light of the above, there exists a present need for improved methods, apparatus, articles-of-manufacture, and coded data signals that reduce the computational complexity of PPM-based data compression. And there exists a present need for such improved methods, apparatus, articles-of-manufacture and coded data signals that permit the cost of the computationally-expensive model building task to be amortized over many compression/decompression cycles and/or scheduled such that it minimizes user-perceptible service disruptions. The invention, as described below and in the accompanying figures, addresses these needs.